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ABSTRACT 

We introduce CTCP, a reliable transport protocol using net- 
work coding. CTCP is designed to incorporate TCP fea- 
tures such as congestion control and reliability while im- 
proving on TCP's performance in lossy and/or dynamic net- 
works. CTCP builds upon the ideas of TCP/NC introduced 
by Sundararajan et al. and uses network coding to provide 
robustness against losses. We provide an implementation 
of CTCP (in userspace) and demonstrate its performance in 
both testbed and production networks. 
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The Transmission Control Protocol (TCP) is one of 
the core protocols of today's Internet Protocol Suite. 
TCP was designed for reliable transmission over wired 
networks, in which losses are generally an indication 
of congestion. This is not the case in wireless net- 
works, where losses are often due to fading, interfer- 
ence, and other physical phenomena. While there has 
been considerable work on improving the operation of 
TCP over wireless networks, particularly in WiFi net- 
works, TCP still performs poorly in many current sys- 
tems. Approaches tailored to specific wireless systems 
may perform very well over the networks for which they 
have been designed but may have limited applicability 
as users move among different WiFi network instances, 
or operate on both WiFi and cellular networks. In gen- 
eral, the performance of wireless networks is highly vari- 
able across different deployments [1 and even within a 
network according to whether the devices are hand-held 
or not [2]. Users may also experience losses with wire- 
line networks, for instance over DSL connections . 

The goal of our design is to keep traditional TCP's 
best features, including congestion control and relia- 
bility, and augment them with network coding for re- 
silience against losses and failures in the network with- 
out explicit consideration of the lower layers. We do 
not adopt a clean slate or physical-layer dependent ap- 
proach that would incorporate physical layer or link 
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layer methods to reduce the types of losses that lead to 
poor performance of TCP. Such methods would gener- 
ally require customization, whereas we seek an approach 
that is widely applicable to different lossy settings. In- 
deed, the measurements we present in our work show 
that we may more than double throughput with our 
approach over a variety of lossy networks, such as a 
testbed with controlled losses, a WiMAX system and a 
sample of public WiFi systems, where we have no ac- 
cess to the routers. Our approach is thus deliberatively 
not transformative but simply meliorative - we seek a 
technique that can be retro-fitted to a variety of existing 
networks as well as bringing benefits in future networks. 

In order to combine the benefits of TCP and network 
coding 0[6], Sundararajan et al. in |7J proposed a new 
protocol called TCP/NC. Packets in TCP/NC use ran- 
dom network coding [8] to single flows. TCP/NC mod- 
ifies TCP's acknowledgment (ACK) scheme so that it 
acknowledges degrees of freedom (dofs) instead of indi- 
vidual packets. This is done by using the concept of seen 
packets. Packets are acknowledged in order, possibly 
before they are decoded, but not before contributing to 
the construction of received packets. Such an approach 
implies that the number of innovative coded packets re- 
ceived, which we term degrees of freedom, is translated 
to the number of consecutive packets received, even be- 
fore packets may have been decoded. Reference [9] pro- 
vides a simple mathematical model and analysis with 
simulation results that show that TCP /NC achieves sig- 
nificantly higher throughput than traditional versions of 
TCP in lossy networks. 

Our protocol, CTCP, builds upon TCP/NC intro- 
duced by [7]- However, we enhance the algorithms from 
TCP/NC to make CTCP more efficient and robust. 
Therefore, CTCP departs from TCP/NC in the follow- 
ing ways. 

1. We implement the design in C for Linux operating 
systems. We implement the protocol in userspace 
(over UDP) for ease of implementation and mod- 
ifications. The implementation demonstrates that 
CTCP is indeed able to achieve high throughput 
despite losses in the network. 

2. CTCP, unlike TCP/NC which introduces network 
coding indirectly through a shim layer between 
TCP and IP layers, is a transport protocol that 
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uses network coding directly. At a first glance, 
this may not be a significant change; however, by 
designing a new transport protocol, we are able to 
leverage network coding more efficiently and intro- 
duce congestion control mechanisms better suited 
to a protocol using network coding. 

3. CTCP is adaptive. TCP/NC assumes a known av- 
erage end-to-end packet loss rate p and determines 
a redundancy factor R ~ for the communica- 
tion [7J. Note that this redundancy factor plays a 
key role in enabling TCP /NC to use network cod- 
ing to overcome losses. However, in a real network, 
p is rarely known a priori and fluctuates over time 
and space. Our protocol estimates p and dynam- 
ically adjusts the redundancy rate R to adjust to 
the losses on the fly. 

4. CTCP uses systematic block coding to manage de- 
lay and complexity. TCP/NC uses a sliding win- 
dow approach for coding operations, which can 
have significant decoding delay at the receiver as 
it may have undesirable worst-case behavior. Fur- 
thermore, the use of a systematic code significantly 
reduces the decoding overhead over random so- 
lutions with dense matrices. When there are no 
losses (p — leading to R ~ 1), CTCP, in effect, 
reduces to a TCP-like protocol without coding. 

5. We provide a congestion control mechanism that 
works well with network coding. Traditional TCP 
uses a sliding transmission window with sequence 
numbers to identify bytes. With coding, any coded 
packet within a block can replace another packet; 
therefore, we modify the congestion control mech- 
anism to use tokens - i.e. a token allows CTCP 
sender to transmit a packet. Our congestion con- 
trol mechanism generates or destroys tokens to ad- 
just CTCP sender's transmission rate. 

This paper discusses the algorithmic and implemen- 
tation details of CTCP and provides extensive exper- 
imentation results to verify its performance. The rest 
of the paper is organized as follows. In Section [2j we 
present some related material. In Section [3j we provide 
an overview of CTCP. In Sections U and [5j we describe 
CTCP sender and receiver, respectively. After describ- 
ing the protocol, in Sections [6] and [Jj we demonstrate 
CTCP's performance and compare it with traditional 
TCP variants in a testbed as well as in real-world pro- 
duction networks, such as WiMAX and publicly avail- 
able WiFi networks. 

2. RELATED WORK 

In order to mitigate the effect of losses on TCP, par- 
ticularly in the context of wireless links, the use of cod- 
ing has been proposed before our work or that of [7J. 
We may roughly taxonomize these into approaches that 



consider the operation of TCP jointly with lower layer 
redundancy, at the physical or MAC level, often in a 
cross-layer approach [10 18J and those that consider re- 
dundancy at the IP layer or above without access to 
the lower layers [T9l425| . Since our goal is to provide 
a system that operates without access to, or even de- 
tailed knowledge of, the physical layer, it is in the lat- 
ter category that our approach belongs, although we 
shall address some issues related to lower layer reliabil- 
ity mechanisms, such as hybrid ARQ (HARQ), in the 
latter part of the paper, when discussing experiments 
over WiMAX systems. 

The approaches in papers implementing coding at 
the IP layer for operation with TCP have generally 
revolved around traditional block codes, often Reed- 
Solomon codes. Sometimes these approaches are com- 
bined with mechanisms such as explicit congestion noti- 
fication (ECN) [H)ll20| . These block-based approaches 
acknowledge packets upon successful decoding and, if 
the number of errors exceeds the predetermined level 
of redundancy, a decoding failure occurs, requiring re- 
transmission of the block. Even though many of the 
approaches at the IP layer or above, often, like the 
work in this paper, consider adaptation to the observed 
losses |18[I24|. the mechanism remains one in which ac- 
knowledgements require decoding of full blocks and re- 
ception of a full block must occur before the window 
can progress. The concept of ACK for seen packets, 
mentioned in the introduction, circumvents the need 
for decoding before acknowledging. 

Moreover, the use of structured codes, in block for- 
mat or otherwise, is not necessary to combat erasures. 
Random network coding |8J can be used to combat ran- 
dom erasures over networks in unicast and multicast 
settings to obtain the optimal rate without any need to 
know where and when erasures occur within the net- 
work [26j . The use of random network coding has been 
demonstrated successfully for broadcast erasure correc- 
tion in wireless networks [27]. The concept of random 
coding with seen-style ACKs has also been proposed for 
delay-tolerant networks (DTNs) in [28]. 

3. OVERVIEW OF CODED TCP (CTCP) 

Before discussing the CTCP sender and receiver in 
detail, we provide a more holistic view of the two here. 

The CTCP sender segments the stream, or the file, 
into a series of blocks as shown in Figure [1] A block 
is chosen to be of a fixed size, equivalent to blksize 
number of packets where each packet is assumed to be 
of fixed length. If the remainder of a file or a stream is 
not large enough to form a complete packet, the packet 
is padded with zeros to ensure that all packets are of the 
same length. A block need not be completely full, i.e. 
a block may have fewer than blksize packets; however, 
block i should be full before block i + 1 is initialized. 
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Table 1: Definitions of the sender parameters 
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Figure 1: CTCP sender divides the data into blocks. 

The CTCP sender keep numblks of blocks in mem- 
ory and the value of numblks should be conveyed to 
the receiver. The value of numblks may be negotiated 
at initialization between the sender and the receiver, 
as numblks directly affects the memory usage on both 
ends. We denote the smallest block in memory to be 
currblk. Note that this does not mean that CTCP 
sender may send numblks x blksize amount of data at 
any given point in time. 

The sender is allowed to transmit packets only if 
the congestion control mechanism allows it to; how- 
ever, whenever it is allowed to transmit, the sender 
may choose to transmit a packet from any one of the 
blocks in memory, i.e. blocks currblk, currblk + 1, 
currblk + numblks — 1. In Section 14.51 we shall dis- 
cuss the sender's algorithm for selecting a block from 
which it sends a packet. The pay load of the transmit- 
ted packet may be coded or uncoded; the details of the 
coding operations can be found in Section WM 

The sender includes the following in each packet: 

• the block number, 

• a seed for a pseudo-random number generator, which 
allows the receiver to generate the coding coeffi- 
cients, 

• the sequence number seqno, and 

• the (coded or uncoded) payload. 

The sequence number for CTCP differs from that of 
TCP - for TCP, a sequence number indicates a spe- 
cific data byte; for CTCP, a sequence number indicates 
that a packet is the seqno-th packet transmitted by the 
sender, thus, is not tied to a byte in the file. 

The CTCP receiver sends acknowledgments (ACKs) 
for the packets it receives. In the ACK, the receiver 
indicates 

• the smallest undecoded block ack _currblk, 

• the number of degrees of freedom (dofs) 
ack_currdof it has received for the current block 
ack_currblk, and 

• the ack _seqno of the packet it is acknowledging. 

Using the information in an ACK, CTCP sender ad- 
justs its behavior. We first describe the sender algo- 
rithm in Section|4l as most of the intelligence of the pro- 
tocol is on the sender's side. Then, we present CTCP 
receiver algorithm in Section [5] CTCP receiver's main 
role is to decode and deliver data to the application. 



Notation 



P 

RTO 



RTT 



seqno _nxt 
seqno _una 
ss_threshold 

time _lastack 

tokens 

blksize 
currblk 

currdof 
numblks 

B (seqno) 
T (seqno) 



Definition 



Short term average packet loss rate 
Retransmission timeout period 
(equal to 7 • RTT where 7 > 1 is a 
constant) 

Current (or the last acknowledged 
packet's) round-trip time 
The minimum round-trip time 
The sequence number of the next 
packet to be transmitted 
The sequence number of the latest 
unacknowledged packet 
Slow-start threshold, i.e. if 
tokens > ss_threshold, the 
sender leaves the slow-start mode 
Timcstamp of when the sender re- 
ceived the latest ACK (initialized to 
the time when the sender receives a 
SYN packet from the receiver) 
Number of tokens, which is concep- 
tually similar to congestion window 
for traditional TCP 
Size of the blocks (in number of 
packets) 

Current block number at the sender, 
which is the smallest unacknowl- 
edged block number 
Number of dofs the receiver has ac- 
knowledged for the current block 
Number of active blocks, i.e. the 
sender may schedule and trans- 
mit packets from blocks currblk, 
currblk+l, currblk+numblks—l 
Block number from which packet 
with seqno was generated from 
Timcstamp of when packet with 
seqno was sent 



4. CTCP SENDER 

We present the sender side algorithm for CTCP. The 
sender maintains several internal parameters (defined in 
Table [TJ) , which it uses to generate coded packets and 
schedule blocks to be transmitted. 

4.1 Network Parameter Estimation 

The CTCP sender estimates the network parameters, 
such as RTT and p, using the ACKs as shown in Al- 
gorithm [1] The sender adjusts its actions, including 
coding operations and congestion control depending on 
these estimates. If the sender does not receive an ACK 
from the receiver for an extended period of time (i.e. 
time-out occurs), the network parameters are reset to 
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Receive an ACK; 

time _lastack <— current time; 

RTT <— time _lastack — T(ack_seqno); 

RTT mm <- minimum of {RTT, RTT min }; 

if ack_currblk > currblk then 

Free blocks currblk, ack _currblk — 1; 
currblk <— ack _currblk; 
currdof <— ack _currdof; 
end 

currdof max{ack_currdof, currdof}] 
if ack_seqno > seqno_una then 

losses <— ack _seqno — seqno _una + 1; 
p^p(l- u y osses+1 + (1 - (1 - u) losses ); 
end 

seqno _una ack _seqno + 1; 

Algorithm 1: CTCP sender algorithm for updating 
the network parameters. Above, /i is the smoothing 
factor used for our modified exponential smoothing 
technique. 



predefined default values. The predefined default values 
may need to be chosen with some care such that they 
estimate roughly what the network may look like. 

The CTCP sender maintains moving averages of p. 
We use a slightly modified version of the exponential 
smoothing technique (for reasons discussed later in this 
paragraph). For p, we can consider the data series we 
are averaging to be a 0-1 sequence, where indicates 
that the packet has been sent successfully and 1 other- 
wise (i.e. a packet loss). Now, assume that there were 
losses number of packets lost. If losses = 0, then the 
update equation for p in Algorithm Q] becomes 



(1) 



where \i is the smoothing factor. If losses = 1, the same 
update equation becomes 

p <- p{l - af + n = (1 - - M) + 0] + M, (2) 

which is identical to executing an exponential smooth- 
ing over two data points (one lost and one acknowl- 
edged). We can repeat this idea for losses > 1 to ob- 
tain the update rule for p in Algorithm [1] Therefore, 
the fact that a single ACK may represent multiple losses 
(losses > 1) leads to a slightly more complicated up- 
date rule for p than that for RTT as shown in Algorithm 
[TJ To the best of our knowledge, such an update rule 
has not been used previously. 

4.2 Reliability 

CTCP achieves reliability by ensuring that each block 
is received and decoded. In Algorithm [TJ CTCP sender 
increments currblk only if it has received an ACK indi- 
cating that the receiver is able to decode currblk - i.e. 



ack _currblk > currblk. This mechanism is equivalent 
to traditional TCP's window sliding scheme in which 
the TCP sender only slides its window when it receives 
an ACK indicating the some bytes have been received. 
In the case of CTCP, the reliability is implemented over 
blocks instead of bytes. 

4.3 Congestion Control Mechanism 

Traditional TCP's AIMD congestion control increases 
TCP sender's congestion window size cwnd by a pack- 
ets per RTT and multiplicatively decreases cwnd by a 
backoff factor j3 on detecting packet losses within one 
RTT inducing a single cwnd backoff. The usual values 
are a = 1 when appropriate byte counting is used, and 
ft = 0.5. On lossy links, repeated backoffs in response 
to noise losses rather than queue overflow can prevent 
cwnd from increasing to fill the available link capacity. 
The behavior is well known and is captured, for exam- 
ple, in |29] in which cwnd scales as y^l.5/p, where p is 
the packet loss rate. 

The basic issue here is that on lossy links, loss is not 
a reliable indicator of network congestion. One option 
might be to use delay, rather than loss, as the indica- 
tor of congestion, but this raises many new issues and 
purely delay-based congestion control approaches have 
not been widely adopted in the internet despite being 
the subject of extensive study. Another option might 
be to use explicit signalling, for example via ECN, but 
this requires both network-wide changes and disabling 
of cwnd backoff on packet loss. These considerations 
motivate consideration of hybrid approaches, making 
use of both loss and delay information. The use of hy- 
brid approaches is well-established, for example Com- 
pound TCP (3D] is widely deployed. 

We consider modifying the AIMD multiplicative back- 
off. Before discussing the details of backoff behavior, 
we emphasize that CTCP uses tokens to control the 
CTCP sender's transmission rate instead of the con- 
gestion window cwnd; therefore, tokens play a similar 
role for CTCP as cwnd does for TCP. A token allows 
the CTCP sender to transmit a packet (coded or un- 
coded). When the sender transmits a packet, the token 
is used. The number of tokens, tokens, is controlled 
according to the modified AIMD multiplicative backoff 
(Algorithm O. 

As shown in Algorithm [21 we modify the AIMD mul- 
tiplicative backoff to have 

a RTT m i n 

^-~RT7^' {3) 
where RTT m i n is the path round-trip propagation delay 
(which is typically estimated as the lowest per packet 
RTT observed during the lifetime of a connection) and 
RTT is the current round-trip time. 

This is similar to the approach considered in |31) . 
which uses (3 = RTT min / RTT max with the aim of mak- 
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if current time > time _lastack + RTO then 
tokens <— initial token number; 
Set to slow-start mode; 
end 

if Receive an ACK on path i then 
if slow-start mode then 
tokens <— tokens + 1; 
if tokens > ss _threshold then 
[ Set to congestion avoidance mode; 
end 
else 

if ack _seqno > seqno _una then 
| tokens <— R1 ^ 1 r p n tokens; 
else 

I tokens <— tokens + 7-7- — ; 

I tokens ' 

end 
end 
end 

Algorithm 2: CTCP sender algorithm for congestion 
control mechanism. 



ing TCP throughput performance less sensitive to the 
level of queue provisioning. Indeed on links with only 
queue overflow losses, ^ reduces to the approach in [3"T] 
since RTT = RTT max (the link queue is full) when 
loss occurs. In this case, when a link is provisioned 
with a bandwidth-delay product of buffering, as per 
standard guidelines, then RTT max = 2RTT min and 
j3 = 0.5, i.e. the behavior is identical to that of stan- 
dard TCP. More generally, when queue overflow occurs 
the sum of the flows' throughputs must equal the link 
capacity B, ^2™ =1 tokenSi/ RTTi — B where n is the 
number of flows. After backoff according to when 
the queue empties then the sum-throughput becomes 
S™=i ftitokensi/ RTT m i n ^ = B. That is, the choice ([3]) 
for f3 decreases the flow's tokens so that the link queue 
just empties and full throughput is maintained. 

On lossy links (with losses in addition to queue over- 
flow losses), use of RTT in adapts j3 to each loss 
event. When a network path is under-utilized, RTT = 
RTT mm (therefore, f3 = 1 and (3 x tokens = tokens). 
Thus, tokens is not decreased on packet loss. Hence, 
tokens is able to grow, despite the presence of packet 
loss. Once the link starts to experience queueing de- 
lay then RTT > RTT mm and j3 < 1, i.e. tokens is 
decreased on loss. Since the link queue is filling, the 
sum-throughput before loss is ^2™ =1 tokenSi/ RTTi — 
B. After decreasing tokens, when the queue empties 
the sum-throughput is at least (when all flows backoff 
their tokens) 53<=i fiitokensi/ RTT m i n ^ = B. That is, 
([3]) adapts j3 to maintain full throughput. 

Although we focus on using §3§ in combination with 
linear additive increase (where a is constant), we note 
that this adaptive backoff approach can also be com- 



bined with other types of additive increase including, 
in particular, those used in Cubic TCP and Compound 
TCP. As shown here, these existing approaches can be 
extended to improve performance on lossy links. 

4. 3. 1 Mathematical Modeling 

In this section, we provide some analysis on our choice 
of (3, the multiplicative backoff factor. 

Consider a link shared by n flows. Let B denote the 
capacity of the link and Ti the round-trip propagation 
delay of flow i. We will assume that the queueing delay 
can be neglected, i.e. the queues are small or the link 
is sufficiently lossy that the queue does not greatly fill. 
We also assume that any differences in the times when 
flows detect packet loss (due to the differences in path 
propagation delay) can be neglected. Let t k denote the 
time of the fc-th network backoff event, where a network 
backoff event is defined to occur when one or more flows 
reduce their tokens. Let Wi(k) denote the tokens of flow 
i immediately before the fc-th network backoff event and 
Sj(fc) = Wi(k)/Ti the corresponding throughput. With 
AIMD we have 



s i (k) = p i (k-l)s i (k-l) + a i T(k) 



(4) 



where on = a/T?, a the AIMD increase rate in pack- 
ets per RTT, T(k) is the time in seconds between the 
k — 1 and fc-th backoff events, f3i(k) is the backoff factor 
of flow i at event k. The backoff factor (3i(k) is a ran- 
dom variable, which takes the value 1 when flow i does 
not experience a loss at network event fc, and otherwise 
takes the value given by ([3]). The time T(k) is also a 
random variable, with distribution determined by the 
packet loss process and typically coupled to the flow 
rates s^(fc), i = 1, • • ■ ,n. 

For example, associate a random variable Sj with 
packet j, where Sj = 1 when packet j is erased and 
otherwise. Assume the Sj are i.i.d with erasure proba- 
bility p. Then Prob(T(k) < t) = 1 - (l-p) N *W where 
N t (k) — 53j = i (t) is the total number of packets 
transmitted over the link in interval t following backoff 
event fc-1 and N t ,i(k) = ft(fc - l)s,(fc - l)t + 0.5q^ 2 is 
the number of packets transmitted by flow i in this in- 
terval t. Also, the probability 7i(fc) := Prob(f3i(k) = 1) 
that flow i does not back off at the fc-th network back- 
off event is the probability that it does see any loss 
during the RTT interval [T(fc), T(fc) +TJ, which can be 
approximated by 7i(fc) = (1 — p) Si ( fe ) Ti on a link with 
sufficiently many flows. 

Since both j3i(k) and T(k) are coupled to the flow 
rates Si(fc), i = 1, ••• ,n, analysis of the network dy- 
namics is generally challenging. However, when the 
backoff factor f3i(k) is stochastically independent of the 
flow rate s^(fc) then analysis is relatively straightfor- 
ward. Note that this assumption is valid in a number 
of useful and interesting circumstances. One such cir- 
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cumstance is when links are loss-free (with only queue 
overflow losses) [35] • Another is on links with many 
flows and i.i.d packet losses, where the contribution of 
a single flow i to the queue occupancy (and so to RTT 
in (|3|)) is small. Further, as we will see later, experi- 
mental measurements indicate that analysis using the 
assumption of independence accurately predicts perfor- 
mance over a range of other network conditions, and so 
results are relatively insensitive to this assumption. 
Given independence, from (|4|). 

E[s t (fc)] = E\fli (*)]E[« 4 (k - 1)] + &iE[T(k)] . (5) 

When the network is also ergodic, a stationary distri- 
bution of flow rates exists. Let E[sj] denote the mean 
stationary rate of flow i. From ([5|) we have 

E[ Si ] = ^-=— E[T], (6) 

1 - E[A] 

Since the factor E[T] is common to all flows, the fraction 
of link capacity obtained by flow i is determined by 
Oi/(l-E|&]). 

Fairness between flows with same RTT: From ([6|) , 
when flows i, j have the same RTT, and so &i = <5y, 
and the same mean backoff factor E[/3;] = E[f3j] then 
they obtain on average the same throughput share. 
Fairness between flows with different RTTs: When 
flows i, j have different round trip times Ti ^ Tj but the 
same mean backoff factor, the ratio of their throughputs 
is E^/E^] = (Tj/Ti) 2 . Observe that this is identical 
to standard TCP behavior |32| . 

Fairness between flows with different loss rates: 

The stationary mean backoff factor E[/3j] depends on 
the probability that flow i experiences a packet loss at 
a network backoff event. Hence, if two flows i and j 
experience different per packet loss rates pi and pj (e.g. 
they might have different access links while sharing a 
common throughput bottleneck) , this will affect fairness 
through E\j3i]. 

Friendliness: The model Q is sufficiently general 
to include AIMD with fixed backoff factor, as used by 
standard TCP. We consider two cases. First, when the 
link is loss-free (the only losses are due to queue over- 
flow) and all flows backoff when the queue fills, then 
f — E[f3i] = 1 — f3i(k). Hence, for a flow i with fixed 
backoff of 0.5 and a flow j with adaptive backoff /3j, 
when the flows have the same RTT the ratio of the 
mean flow throughputs is E[sj]/E[sj] = 2(1 — f3j) by 
©. When /3j = Tj/RTT 3 = 0.5 then the throughputs 
are equal. Since RTTj = Tj+q max /B where q max is the 
link buffer size and B the link rate, then /3j = 0.5 when 
Qmax = BTj, i.e. the buffer is size at the bandwidth- 
delay product. The second case we consider here is 
when the link has i.i.d packet losses with probability p. 
When p is sufficiently large that the queue rarely fills, 
then queue overflow losses are rare and the throughput 



of a flow i with fixed backoff of 0.5 is accurately mod- 
eled by the Padhye model [53] ■ That is, the throughput 
is largely decoupled from the behavior of other flows 
sharing the link (since coupling takes place via queue 
overflow) and, in particular, this means that flows us- 
ing adaptive backoff do not penalize flows which use 
fixed backoff. We present experimental measurements 
confirming this behavior in Section [6.31 

4.4 Network Coding Operations 

The coding operations are performed over blocks (Fig- 
ure [J). Unlike TCP/NC [7], we do not use a sliding 
window for coding operations. The main reason behind 
this design decision is for delay and complexity. The 
sliding window approach allows for better throughput 
performance; however, when using this approach, the 
receiver may not be able to decode even the first packet 
of the file until the entire file is received. As a result, 
the decoding complexity may be high, as the decoding 
operation may be performed over the entire file (instead 
of segments of the file). This may not be a significant 
concern for small file transfers; however, for some ap- 
plications such as multimedia streaming and large file 
transfers, this may be a significant concern. Therefore, 
in our design, we have opted to use block codes, where 
we can bound the delay and the complexity by changing 
the block size blksize. 

Setting blksize = 1 leads to operations similar to 
that of traditional TCP variants and cannot effectively 
take advantage of network coding. On the other hand, 
setting blksize too large leads to the problems faced 
by TCP/NC. Therefore, blksize has to be chosen with 
care. In our experience, it is desirable to set blksize to 
be similar to the bandwidth x delay of the network. This 
is because larger block size yields to higher efficiency at 
the cost of higher delay, and a balance needs to be struck 
between these competing requirements. 

The coding field size has been chosen with some care 
as well. Similar to block size, using higher field size 
leads to higher probability of generating independent 
dofs (leading to efficiency); however, this comes at the 
cost of coding and decoding complexity. We use a field 
of F256, i.e. each coefficient is a single byte, as it has 
been shown to provide good performance [TJ. 

To ensure that coding is only performed when nec- 
essary, we use systematic block codes - i.e. uncoded 
packets are transmitted before coded packets are sent. 
In generating coded packets, there are many options. 
The sender may only code a subset of the packets in a 
block. In our design, we use a simple approach - a coded 
packet is generated by randomly coding all packets in 
the block. This approach is most effective in terms era- 
sure correction. With high probability, a coded packet 
will correct for any single erasure in the block. 

4.5 Transmission and Block Scheduling 
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Initialize an array onfly\\ to 0; 

for seqno in [seqno_una, seqno _nxt — 1] do 

if current time < T(seqno) + 1.5RTT then 
| on fly[B '(seqno)} on f ly[B (seqno)} + 1; 

end 
end 

for blkno in [currblk, currblk + numblks — 1] do 
if blkno = currblk and 

(1 — p)onfly[currblk] < blksize — currdof then 

Transmit a packet with sequence number 

seqno _nxt from block blkno; 

seqno _nxt <— seqno _nxt + 1; 
else if (1 — p) on fly [blkno] < blksize then 

Transmit a packet with sequence number 

seqno _nxt from block blkno; 

seqno _nxt 4— seqno _nxt + 1; 
end 
end 

Algorithm 3: CTCP sender algorithm for block 
scheduling when a token is available. 



When a token is available, the CTCP sender decides 
which block to transmit a packet from. The block schedul- 
ing algorithm (Algorithm [3]) plays a key role in CTCP's 
operations. The algorithm first computes the number 
of packets in transit from the sender to the receiver. 
Given p, the sender can compute the expected number 
of packets the receiver will receive for any given block. 
In determining the expected number of dofs the receiver 
will receive for any given block, we exclude the packets 
that have been transmitted more than 1.5 • RTT time 
ago, as they are likely to be lost or significantly delayed. 
The constant factor of 1.5 may be adjusted depending 
on the delay constraints of the application of interest; 
however, the constant factor should be > 1. 

The goal of the sender is to ensure that, in expec- 
tation, the receiver will receive enough packets to de- 
code the block. The sender prioritizes block i before 
i + 1; therefore, currblk is of the highest priority. Note 
that the algorithm treats currblk slightly differently 
from the rest of the blocks. In our design, the CTCP 
receiver informs the sender of how many dofs it has 
received (currdof) for block currblk. Therefore, the 
sender is able to use the additional information to de- 
termine more precisely whether another packet should 
be sent from block currblk or not. It is not difficult 
to piggy-back more information on the ACKs. For ex- 
ample, we could include how many dofs the receiver 
has received for blocks currblk as well as currblk + 1, 
currblk + 2, currblk + numblks — 1. However, for 
simplicity, the CTCP receiver only informs the sender 
the number of dofs received for block currblk. 

In Algorithm [3j we assume that all blocks are of 



cc <— coding coefficients of the received packet; 
pp <— pay load of the received packet; 
index <— index of the first non-zero element in cc; 
if Cbikno[index, :] is empty then 

pivot <— value of c at index index; 

Insert cc/ pivot into Cukno[index, :}; 

Insert pp/ pivot into Pukno[index , :]; 

return TRUE; 
else 

if index < blksize then 

pivot 4— value of c at index index; 

cc <— cc — pivot ■ Cukno[index, :]; 

pp <- pp- pivot ■ Pukno [index, :] ; 

pivot <— value of cc at index index + 1; 

cc «— c/pivot; 

pp <— p/pivot; 

if cc ^ then 

Recursively call itself with updated cc 
and pp; 

end 
end 

return FALSE; 
end 

Algorithm 4: CTCP receiver algorithm for updating 
Cbikno and Pukno when a packet from block blkno is 
received. We denote cc to be the coding coefficients 
and pp the (coded) payload of the received packet. 

length blksize. We note that CTCP can cope with 
blocks of varying length; however, for simplicity of pre- 
sentation, we have chosen to present the algorithms 
with a fixed block length. 

5. CTCP RECEIVER 

We now present the receiver side algorithm for CTCP. 
The receiver is responsible for decoding the received 
data. Another important role of the receiver is to con- 
struct ACKs for the sender. Whenever the receiver re- 
ceives a packet, it needs to check whether the current 
block is decodable (ack _currblk) and how many dofs 
it has received for the current block (ack _currdo f) . 

5.1 Decoding Operations 

The CTCP receiver also organizes the received pack- 
ets into blocks. For each block blkno, the receiver ini- 
tializes a blksize x blksize matrix Cukno for the cod- 
ing coefficients and a corresponding payload structure 
Pukno- Whenever a packet from blkno is received, the 
coding coefficients and the coded payload are inserted 
to Cukno and Pukno respectively as shown in Algorithm 
|4j Algorithm [4] returns FALSE if the packet is linearly 
dependent to the previously received packets; otherwise 
it returns TRUE. Note that Algorithm [4] ensures that 
Cbikno is an upper-triangular matrix with diagonal en- 
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Figure 2: Schematic of experimental testbed. 

tries equal to one. Since CTCP sender uses a systematic 
code, CTCP receiver may often be able to insert pp and 
cc directly - i.e. row index of Cukno is empty. 

When Algorithm 0] returns TRUE, the receiver sets 
ack_currdof <— ack _currdof + 1. If ack_currdof 
is equal to blksize, then the receiver acknowledges that 
enough dofs have been received for ack _currblk and up- 
date ack _currblk ack_currblk + 1 (ack _currdof is 
also reset to reflect the dofs needed for the new ack _currblk) 
If Algorithm Q] returns FALSE, then the receiver trans- 
mits an ACK (corresponding to the packet received); 
however, it does not update ack _currdof nor ack _currblk. 

Once enough dofs are received for a block, the receiver 
can decode all packets within the block. This results in 
performing a Gauss- Jordan elimination on an upper- 
triangular matrix Cukno and its corresponding Pukno- 

6. EXPERIMENTAL MEASUREMENTS 

In this section, we demonstrate CTCP's performance 
in a testbed. We present results on not only throughput 
but also on friendliness and fairness. 

6.1 Testbed Setup 

The lab testbed consists of commodity servers (Dell 
Poweredge 850, 3GHz Xeon, Intel 82571EB Gigabit NIC) 
connected via a router and gigabit switches (Figure [5]). 
Sender and receiver machines used in the tests both run 
a Linux 2.6.32.27 kernel. The router is also a commod- 
ity server running FreeBSD 4.11 and ipf w-dummynet. 
It can be configured with various propagation delays T, 
packet loss rates p, queue sizes Q and link rates B to 
emulate a range of network conditions. As indicated 
in Figure [21 packet losses in dummynet occur before the 
rate constraint, not after, and so do not reduce the bot- 
tleneck link capacity B. Unless otherwise stated, ap- 
propriate byte counting is enabled for standard TCP 
and experiments are run for at least 300s. Data traffic 
is generated using rsync (version 3.0.4), HTTP traffic 
using apache2 (version 2.2.8) and wget (version 1.10.2), 
video traffic using vie as both server and client (version 
0.8.6e as server, version 2.0.4 as client). 

CTCP is implemented in userspace as a forward proxy 
located on the client and a reverse proxy located on the 
server. This has the advantage of portability and of 
requiring neither root-level access nor kernel changes. 
Traffic between the proxies is sent using CTCP. With 
this setup, a client request is first directed to the local 
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Figure 3: Measurements of goodput efficiency against 
packet loss rate, link rate and RTT. The Theory curve 
in Figure I3bl is generated using Equation ([7} . 

forward proxy. This transmits the request to the re- 
verse proxy, which then sends the request to the appro- 
priate port on the server. The server response follows 
the reverse process. The proxies support the SOCKS 
protocol and standard tools allow traffic to be trans- 
parently redirected via the proxies. In our tests, we 
used proxychains (version 3.1) for this purpose. 

6.2 Efficiency 

Figure [3] presents experimental measurements of the 
efficiency (equal to goodput s f standard TC p and 

J \ ^ link capacity ' 

CTCP over a range of network conditions. Figure |3a| 
shows the measured efficiency versus the packet loss 
probability p for a 25Mbps link with 25ms RTT and 
a bandwidth-delay product of buffering. Baseline data 
is shown for standard TCP (i.e. TCP SACK/Reno), 
Cubic TCP (current default on most Linux distribu- 
tions) , H-TCP, together with the value y/l.h/p packets 
per RTT predicted by the popular Padhye model [29 . 
It can be seen that the measurements for standard TCP 
are in good agreement with the Padhye model, as ex- 
pected. Also that Cubic TCP and H-TCP closely fol- 
low standard TCP behavior, again as expected since the 
link bandwidth-delay product of 52 packets lies in the 
regime where these TCP variants seek to ensure back- 
ward compatibility with standard TCP. Observe that 
the achieved goodput decreases rapidly with increasing 
loss rate, falling to 20% of the link capacity when the 
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packet loss rate is 1%. This feature of standard TCP is, 
of course, well known. Compare this, however, with the 
efficiency measurements for CTCP, which are shown in 
Figure [3a| and also given in more detail in Figure I3bl 
The goodput is > 96% of link capacity for a loss rate of 
1%, a roughly five-fold increase in goodput compared 
to standard TCP. 

Figure [3bl presents the efficiency of CTCP for a range 
of link rates, RTTs and loss rates. It shows that the 
efficiency achieved is not sensitive to the link rate or 
RTT. Also shown in Figure [3b] is a theoretical upper 
bound on the efficiency calculated using 

fc=0 ^ ' 

where N — 32 is the block size, p the packet erasure 
probability and n — \_N/(1 — p)\ — N is the number 
of forward-transmitted coded packets sent with each 
block. This value n is the mean number of such forward- 
transmitted coded packets that are unnecessary (be- 
cause there are fewer then n erasures). 

The efficiency achieved by CTCP is also insensitive 
to the buffer provisioning, as discussed in Section l4~3l 
This property is illustrated in Figure [3a| which presents 
CTCP measurements when the link buffer is reduced 
in size to 25% of the bandwidth-delay product. The 
efficiency achieved with 25% buffering is close to that 
with a full bandwidth-delay product of buffering. 

6.3 Friendliness with Standard TCP 

Figuresg]and[5]confirm that standard TCP and CTCP 
can coexist in a well-behaved manner. In these measure- 
ments, a standard TCP flow and a CTCP flow share 
the same link competing for bandwidth. As a baseline, 
Figure 0] presents the goodputs of TCP and CTCP for 
range of RTTs and link rates on a loss-free link (i.e. 
when queue overflow is the only source of packet loss). 
As expected, it can be seen that the standard TCP and 
CTCP flows consistently obtain similar goodput. 

Figure O presents goodput data when the link is lossy. 
The solid lines indicate the goodputs achieved by the 
CTCP flow and the standard TCP flow sharing the 
same link with varying packet loss rates. At low loss 
rates, they obtain similar goodputs; but as the loss 
rate increases, the goodput of standard TCP rapidly 
decreases (as already observed in Figure I5aj) . 

For comparison, in Figure [5l we also show (using the 
dotted lines) the goodput achieved by a standard TCP 
flow when competing against another standard TCP 
flow (i.e. when two standard TCP flows share the link). 
Note that the goodput achieved by a standard TCP flow 
(dotted line) when competing against another standard 
TCP flow is close to that achieved when sharing the link 
with a CTCP flow (solid line). This demonstrates that 
CTCP does not penalize the standard TCP flow. 
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Figure 4: Goodput for a standard TCP and a CTCP 
flow sharing a loss-free link; results are shown for 
10Mbps and 25Mbps links with varying RTTs. 
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(a) Lossy 10Mbps link with (b) Lossy 25Mbps link with 
RTT=25ms RTT=25ms 



Figure 5: Goodput against link loss rate for (i) a TCP 
and a CTCP flow sharing this link (solid lines), and (ii) 
two TCP flows sharing lossy link (dashed line). 

6.4 Fairness among CTCP Flows 

We turn now to fairness, i.e. how goodput is allo- 
cated between competing CTCP flows. Figure [6al plots 
a typical tokens time history for two CTCP flows shar- 
ing a lossy link. The second flow (black) is started after 
the first (grey) so that we can observe the convergence 
to fairness. It can be seen that the two flows' tokens 
rapidly converge. Figure [6bl presents corresponding good- 
put measurements for a range of link rates, RTTs, and 
loss rates. Again, the two CTCP flows consistently 
achieve similar goodputs. 

6.5 Application Performance 

In this section, we present our testbed results seen by 
various applications. 

6.5.1 Web 

Figure [7] shows measurements of HTTP request com- 
pletion time against file size for standard TCP and 
CTCP. The HTTP requests are generated using wget 
and the response is by an apache2 web server. Note the 
log scale on the y-axis. 

The completion times with CTCP are largely insensi- 
tive to the packet loss rate. For larger file sizes, the com- 
pletion times approach the best possible performance 
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Figure 6: Measurements of tokens and goodput for two 
CTCP flows sharing a lossy link. Figure [5a] provides a 
sample of tokens time history while Figure [6b] summa- 
rizes the goodputs for a range of packet loss rates (%) 
with 10Mbps and 25Mbps links (RTT 25ms). Similar 
behavior was observed for other values of RTTs. 
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Figure 7: Measured HTTP request mean completion 
time against file size over 25Mbps link with RTT = 
10ms. Data is shown for standard TCP (red) and 
CTCP (black) for a range of loss rates. Error bars are 
comparable in size to the symbols used in the plot and 
so are omitted. 

indicated by the dashed line. For smaller file sizes, the 
completion time is dominated by slow-start behavior. 
Note that CTCP and TCP achieve similar performance 
when the link is loss-free; however, TCP's completion 
time quickly increases with loss rate. For a 1MB connec- 
tion, the completion time with standard TCP increases 
from 0.9s to 18.5s as the loss rate increases from 1% to 
20%, while for a 10MB connection the corresponding 
increase is from 7.1s to 205s. 

6.5.2 Streaming Video 
Figure[5]plots performance measurements for stream- 
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Figure 8: Measurements of video streaming perfor- 
mance against loss rate with a 25Mbps link and a RTT 
of 10ms. Data is shown for standard TCP and CTCP. 
Figure [8a| shows the running time taken to play a video 
of nominal duration (60s); Figure [8b] shows the number 
of under-runs of the playout buffer at the client. 

ing video for a range of packet loss rates on a 25Mbps 
link with RTT equal to 10 ms. A vie server and client 
are used to stream a 60s video. Figurel8alplots the mea- 
sured time for playout of the video to complete. Again, 
note the log scale on the y-axis. 

The playout time with CTCP is approximately 60s 
and is insensitive to the packet loss rate. In contrast, 
the playout time with standard TCP increases from 60s 
to 95s when the loss rate is increased from 0% to 1%, 
and increases further to 1886s (31 minutes) as the loss 
rate is increased to 20%. Figure [8b] plots measurements 
of playout buffer under-run events at the video client. 
It can be seen that there are no buffer under-run events 
when using CTCP even when the loss rate is as high as 
20%. With standard TCP, the number of buffer under- 
runs increases with loss rate until it reaches a plateau at 
around 100 events, corresponding to a buffer underrun 
occurring after every playout of a block of frames. In 
terms of user experience, the increases in running time 
result in the video repeatedly stalling for long periods 
of time and are indicative of a unsatisfactory quality of 
experience even at a loss rate of 1%. 

7. REAL- WORLD PERFORMANCE 

In this section, the performance of CTCP and TCP 
are measured in production networks to the determine 
real- world gains of the new protocol. Specifically, exper- 
iments were executed on both an IEEE 802.16 cellular 
network [33J and several public WiFi networks. 

7.1 IEEE 802.16 WiMAX Network 

7.1.1 Experiment Setup 

The IEEE 802.16 WiMAX network used was made 
available through the Global Environment for Network 
Innovations (GENI) collaborative research framework 
[34] . Experiments were run using a single WiMAX base 
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Table 2: IEEE 802.16 BS and SS Configurations. 



BS 


SS 


Tx. Power 


CINR 


RSSI 


Tx. Power 


21 dBm 


20 dB 


-71 dBm 


-63 dBm 


22 dBm 


21 dB 


-70 dBm 


-63 dBm 


23 dBm 


22 dB 


-69 dBm 


-63 dBm 


24 dBm 


23 dB 


-68 dBm 


-63 dBm 


25 dBm 


23 dB 


-68 dBm 


-63 dBm 


26 dBm 


24 dB 


-67 dBm 


-63 dBm 



station (BS) acting as the server and a single WiMAX 
subscriber station (SS) acting as the client (both run- 
ning a Ubuntu 10.10 kernel). The downlink modula- 
tion and coding scheme (MCS) for every experiment 
remained constant (64-QAM CTC 5 /6), while the trans- 
mit power levels were adjusted to obtain different packet 
loss rates and simulate varying distances between the 
WiMAX BS and SS. The BS transmit power, and cor- 
responding SS signal to noise ratios (SNR), were chosen 
to reflect those that would be observed in deployed cel- 
lular networks. While increasing the BS transmit power 
will result in fewer losses, it would not necessarily reflect 
a typical cellular network experience. 

The average transmit (Tx) power, Carrier to Interfer- 
ence plus Noise Ratio (CINR), Received Signal Strength 
Indication (RSSI), and MCS used are shown in Tabled 
The uplink MCS and Tx power was held constant at 64- 
QAM, CTC 1/2 and -63 dBm respectively. Automatic 
Repeated reQuest (ARQ) and Hybrid-ARQ (HARQ) 
are both turned off to avoid any effects on the through- 
put and delay as a result of these algorithms. A com- 
prehensive study of the interaction between HARQ and 
ARQ in 802.16 networks with coding is provided in |35| . 

The same version of CTCP as in Section |5] was used; 
however, we introduce two changes. First, instead of 
using a series of proxies to redirect traffic, a simple ftp 
program was generated to send data over CTCP. Sec- 
ond, a modification to the congestion control mecha- 
nism was made for some of the experiments so that 
RTT m i n is reset to the last RTT measurement taken if 
token < 8. The reason for this change will be explained 
in the next subsection. All data, generated using iperf 
(version 2.0.4), was sent using the operating system's 
default TCP implementation (i.e., Cubic TCP). Exper- 
iments were run for each of the BS Tx power levels 
indicated in Table H standard TCP and CTCP data 
transfers were executed individually without any addi- 
tional traffic on the network, and all experiments were 
run for a minimum of 300 s. 

7.7.2 Results and Discussion 

The WiMAX network provides insight into the per- 
formance of CTCP in a production cellular network. 
Specifically, these networks experience a wide range of 
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Figure 9: Measured WiMAX sample distributions for 
packet loss rate and RTT. The center line represents 
the median, and the top/bottom of the box represents 
the 75th/25th percentiles respectively. The whiskers 
extend to the most extreme data points not considered 
outliers and outliers are plotted individually. 
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Figure 10: TCP's and the modified CTCP's goodput 
with a WiMAX BS Tx power of 21 dBm. 
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Figure 11: Mean goodput as a function of the mean 
packet loss probability for the WiMAX network. 

packet loss rates and RTT jitter for any given SNR. Fig- 
ure IH1 provides the distribution of packet loss rate and 
RTT measured. 

A sample of the results is shown in Figure [TUl The 
BS Tx power was set to 21 dBm. The black line is the 
measured goodput for the modified CTCP version, and 
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(a) CTCP Time = 313 s, (b) CTCP Time = 388 s,(c) CTCP Time = 676 s,(d) CTCP Time = 292 s,(e) CTCP Time = 1093 s, 

TCP Time = 807 s, TCP Time = 1151 s, TCP Time = 753 s, TCP Time = 391 s, TCP Time = 3042 s, 

Mean PLR = 4.28%, Mean PLR = 5.25%, Mean PLR = 4.65%, Mean PLR = 4.56%, Mean PLR = 2.16%, 

Mean RTT = 54.21 ms Mean RTT = 73.51 ms Mean RTT = 106.31 ms Mean RTT = 50.39 ms Mean RTT = 208.94 ms 



Figure 12: Public WiFi Network Test Traces (CTCP in black, TCP in red). The download completion times, the 
mean packet loss rate (PLR) , and mean RTT for each experiment are also provided. 



the red line is the standard TCP goodput. The average 
packet loss rate for the trace shown is 0.3238. The mean 
standard TCP goodput and mean CTCP goodput are 
0.09917 Mbps and 0.5148 Mbps, respectively; resulting 
in a gain of approximately 5 times that of standard 
TCP. In addition, CTCP's mean efficiency was 0.9406, 
which is consistent with the results presented earlier. 
Finally, the mean goodput measurements for all BS Tx 
power settings are shown in Figure [TO 

These figures show several interesting results. First, 
the spikes in Figure [TU] at approximately 200 and 250 
seconds indicate CTCP decoding events. Since the loss 
rate is approximately 0.3 when Tx power is 21 dBm, 
CTCP introduces redundant packets. Prior to these 
spikes, the number of blocks (therefore, packets) that 
cannot be decoded is fairly large. Upon receipt of enough 
dofs from the server, the client can decode and pass a 
large number of packets up to the application layer, re- 
sulting in the spikes shown in Figure 1101 

Second, the unmodified CTCP gains with respect to 
standard TCP, shown in Figure [TTJ are not nearly as 
large as those shown in Figure [5j The gain is reduced 
because of the RTT jitter rather than the capacity of 
the network (the network capacity has been measured 
to be greater than 10 Mbps for all Tx power settings). 
As shown in Figure |H1 the majority of measured RTT 
samples are approximately 84 ms for Tx powers less 
than 25 dBm, while the sample values ranged widely 
from 26 ms to 264 ms. This variability is not necessar- 
ily a result of congestion but rather the 802.16 MAC. 
When an RTT sample less than 84 ms is measured, 
CTCP's RTT m in is updated to a value that is not nec- 
essarily the base RTT, which should represent the true 
path delay. Because tokens is reduced by RT ^p^ n every 
time a packet is lost, tokens reduced to its minimum 
quickly under high loss rates resulting in a significantly 
lower throughput. In order to counteract the effects of 
RTT jitter on CTCP, a small modification to the con- 
gestion control algorithm was introduced specifically for 
the WiMAX tests. This change resets RTT min to the 
last RTT measurement when tokens drops below the 
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Figure 13: Mean goodput for each of the experiments 
shown in Figure [T2l 

initial tokens value (i.e., tokens < 8). This approach 
does not necessarily increase the goodput of short lived 
sessions, but does benefit the goodput of longer sessions 
by approximately twice that of the unmodified CTCP. 

When comparing the gains we observed with those 
presented in |35| . we find that introducing network cod- 
ing at the transport layer alone is not necessarily the 
best strategy for increasing throughput; but it does of- 
fer a level of flexibility that cannot be achieved by [35] . 
In [35] . network coding was introduced just above the 
MAC layer and was shown to provide throughput gains 
of up to 5.9 (for UDP traffic). Their implementation 
aims to improve throughput over a single WiMAX link 
and requires changes to the WiMAX BS, which does 
not provide a complete end-to-end solution and is more 
difficult to deploy. However, our results, as well as [35] . 
indicate that including network coding throughout the 
network stack may further increase throughput. 

7.2 Public WiFi Network 

In addition to WiMAX, we conducted several ex- 
periments over public WiFi networks. A 50 MB file 
was downloaded from a server (running Ubuntu 10.04.3 
LTS) located on the MIT campus to a laptop (running 
Ubuntu 12.04.1 LTS) using various public WiFi net- 
works in the greater Boston area. We use the simple 
ftp program described in Section I7TT1 to generate and 
send data over CTCP, and Linux's native secure copy 
program (scp) to test the standard TCP performance. 

Figure [T2l shows the traces for five of the experiments. 
It is important to point out that standard TCP stalled 
and had to be restarted twice before successfully com- 
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pleting in the test shown in Figure I12cl CTCP, on the 
other hand, never stalled nor required a restart. 

Each trace represents a different WiFi network that 
was chosen because of the location, accessibility, and 
perceived congestion. For example, the experiments 
were run over WiFi networks in shopping center food 
courts, coffee shops, and hotel lobbies. In Figures I12al 
- I12d[ the WiFi network spanned a large user area in- 
creasing the possibility of hidden terminals; a scan of 
most of the networks showed > 40 active WiFi radios, 
which also increases the probability of collisions. The 
only experiment that had a small number of terminals 
(i.e. five active radios) is shown in Figure [T^e] 

In each of the experiments, CTCP achieved a larger 
average goodput and faster completion time. The av- 
erage throughput for both CTCP and TCP is shown in 
Figure [T31 Taking the mean throughput over all of the 
conducted experiments, CTCP achieves a goodput of 
approximately 750 kbps while standard TCP achieves 
approximately 300 kbps; resulting in a gain of approx- 
imately 2.5. If we take into account the mean packet 
loss rate of approximately 4%, the gain presented here 
is similar to that of the WiMAX experiments presented 
in Section [7Tl Finally, we note that in several of the ex- 
periments, CTCP experienced fewer timeouts and was 
able to maintain higher throughput than TCP. 

We emphasize the observed loss rates of approximately 
4% in Figure 1121 which is quite high and unexpected; 
resulting in CTCP's significant performance gain over 
TCP's. We believe that the loss rate is not only due 
to randomness but also due to congestion, interference, 
and hidden terminals. This is an area that would be 
worthwhile to investigate further. If our intuition is in- 
deed correct, we believe that CTCP can greatly help 
increase efficiency in challenged network environments. 

8. CONCLUSIONS AND FUTURE WORK 

We proposed CTCP, a reliable transport protocol that 
uses network coding. We implemented CTCP within 
the application layer (over UDP). As a result, our im- 
plementation requires neither kernel level modifications 
nor changes to the network's internal nodes, which may 
allow CTCP to be more easily deployed. We further 
showed that CTCP performs significantly better than 
standard TCP, especially in lossy networks. 

There are areas for further research. CTCP's conges- 
tion control mechanism performs well in lossy networks 
where an increase in RTT indicates congestion. When 
the RTT also varies with non-congestion events such 
as variability resulting from specific MAC implementa- 
tions, as shown in 17.11 CTCP's congestion control can 
needlessly limit throughput. New approaches for fair- 
ness and friendliness may be needed for such networks. 
In addition, we did not investigate the potential impact 
of active queue management (AQM) on CTCP. How- 



ever, the effect of AQM may not be significant as fewer 
networks use AQMs with the introduction of protocols 
using streamlets, selective repeat mechanisms, and new 
congestion control mechanisms such as Cubic. Another 
possible extension is to allow re-encoding within the 
network [8 , 26, 36J , although this may require changes 
within the network (not just end-to-end). However, this 
approach has shown to increase efficiency. Finally, we 
believe that CTCP can be extended to provide gains in 
multi-path environments |37) . By coding over multiple 
paths, initial simulations and experiments in |37| show 
that we can achieve the sum rate of each path without 
the complexity of tracking and scheduling individual 
packets over each path. 
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